{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Redispatch Example with SciGRID network\n",
    "\n",
    "In this example, we compare a 2-stage market with an initial market clearing in two bidding zones with flow-based market coupling and a subsequent redispatch market (incl. curtailment) to an idealised nodal pricing scheme."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "import pypsa\n",
    "import matplotlib.pyplot as plt\n",
    "import cartopy.crs as ccrs\n",
    "from pypsa.descriptors import get_switchable_as_dense as as_dense"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "solver = \"cbc\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "## Load example network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "o = pypsa.examples.scigrid_de(from_master=True)\n",
    "o.lines.s_max_pu = 0.7\n",
    "o.lines.loc[[\"316\", \"527\", \"602\"], \"s_nom\"] = 1715\n",
    "o.set_snapshots([o.snapshots[12]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n = o.copy()  # for redispatch model\n",
    "m = o.copy()  # for market model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "o.plot();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Solve original nodal market model `o`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let us solve a nodal market using the original model `o`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "o.optimize(solver_name=solver)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Costs are 301 k€."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build market model `m` with two bidding zones"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this example, we split the German transmission network into two market zones at latitude 51 degrees.\n",
    "\n",
    "You can build any other market zones by providing an alternative mapping from bus to zone. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "zones = (n.buses.y > 51).map(lambda x: \"North\" if x else \"South\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we assign this mapping to the market model `m`.\n",
    "\n",
    "We re-assign the buses of all generators and loads, and remove all transmission lines within each bidding zone.\n",
    "\n",
    "Here, we assume that the bidding zones are coupled through the transmission lines that connect them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for c in m.iterate_components(m.one_port_components):\n",
    "    c.df.bus = c.df.bus.map(zones)\n",
    "\n",
    "for c in m.iterate_components(m.branch_components):\n",
    "    c.df.bus0 = c.df.bus0.map(zones)\n",
    "    c.df.bus1 = c.df.bus1.map(zones)\n",
    "    internal = c.df.bus0 == c.df.bus1\n",
    "    m.mremove(c.name, c.df.loc[internal].index)\n",
    "\n",
    "m.mremove(\"Bus\", m.buses.index)\n",
    "m.madd(\"Bus\", [\"North\", \"South\"]);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can solve the coupled market with two bidding zones."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m.optimize(solver_name=solver)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Costs are 214 k€, which is much lower than the 301 k€ of the nodal market.\n",
    "\n",
    "This is because network restrictions apart from the North/South division are not taken into account yet."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can look at the market clearing prices of each zone:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m.buses_t.marginal_price"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build redispatch model `n`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, based on the market outcome with two bidding zones `m`, we build a secondary redispatch market `n` that rectifies transmission constraints through curtailment and ramping up/down thermal generators.\n",
    "\n",
    "First, we fix the dispatch of generators to the results from the market simulation. (For simplicity, this example disregards storage units.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p = m.generators_t.p / m.generators.p_nom\n",
    "n.generators_t.p_min_pu = p\n",
    "n.generators_t.p_max_pu = p"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, we add generators bidding into redispatch market using the following assumptions:\n",
    "\n",
    "* All generators can reduce their dispatch to zero. This includes also curtailment of renewables.\n",
    "* All generators can increase their dispatch to their available/nominal capacity.\n",
    "* No changes to the marginal costs, i.e. reducing dispatch lowers costs.\n",
    "\n",
    "With these settings, the 2-stage market should result in the same cost as the nodal market."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "g_up = n.generators.copy()\n",
    "g_down = n.generators.copy()\n",
    "\n",
    "g_up.index = g_up.index.map(lambda x: x + \" ramp up\")\n",
    "g_down.index = g_down.index.map(lambda x: x + \" ramp down\")\n",
    "\n",
    "up = (\n",
    "    as_dense(m, \"Generator\", \"p_max_pu\") * m.generators.p_nom - m.generators_t.p\n",
    ").clip(0) / m.generators.p_nom\n",
    "down = -m.generators_t.p / m.generators.p_nom\n",
    "\n",
    "up.columns = up.columns.map(lambda x: x + \" ramp up\")\n",
    "down.columns = down.columns.map(lambda x: x + \" ramp down\")\n",
    "\n",
    "n.madd(\"Generator\", g_up.index, p_max_pu=up, **g_up.drop(\"p_max_pu\", axis=1))\n",
    "\n",
    "n.madd(\n",
    "    \"Generator\",\n",
    "    g_down.index,\n",
    "    p_min_pu=down,\n",
    "    p_max_pu=0,\n",
    "    **g_down.drop([\"p_max_pu\", \"p_min_pu\"], axis=1)\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's solve the redispatch market:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n.optimize(solver_name=solver)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And, as expected, the costs are the same as for the nodal market: 301 k€."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can plot both the market results of the 2 bidding zone market and the redispatch results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axs = plt.subplots(\n",
    "    1, 3, figsize=(20, 10), subplot_kw={\"projection\": ccrs.AlbersEqualArea()}\n",
    ")\n",
    "\n",
    "market = (\n",
    "    n.generators_t.p[m.generators.index]\n",
    "    .T.squeeze()\n",
    "    .groupby(n.generators.bus)\n",
    "    .sum()\n",
    "    .div(2e4)\n",
    ")\n",
    "n.plot(ax=axs[0], bus_sizes=market, title=\"2 bidding zones market simulation\")\n",
    "\n",
    "redispatch_up = (\n",
    "    n.generators_t.p.filter(like=\"ramp up\")\n",
    "    .T.squeeze()\n",
    "    .groupby(n.generators.bus)\n",
    "    .sum()\n",
    "    .div(2e4)\n",
    ")\n",
    "n.plot(\n",
    "    ax=axs[1], bus_sizes=redispatch_up, bus_colors=\"blue\", title=\"Redispatch: ramp up\"\n",
    ")\n",
    "\n",
    "redispatch_down = (\n",
    "    n.generators_t.p.filter(like=\"ramp down\")\n",
    "    .T.squeeze()\n",
    "    .groupby(n.generators.bus)\n",
    "    .sum()\n",
    "    .div(-2e4)\n",
    ")\n",
    "n.plot(\n",
    "    ax=axs[2],\n",
    "    bus_sizes=redispatch_down,\n",
    "    bus_colors=\"red\",\n",
    "    title=\"Redispatch: ramp down / curtail\",\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also read out the final dispatch of each generator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grouper = n.generators.index.str.split(\" ramp\", expand=True).get_level_values(0)\n",
    "\n",
    "n.generators_t.p.groupby(grouper, axis=1).sum().squeeze()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Changing bidding strategies in redispatch market"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also formulate other bidding strategies or compensation mechanisms for the redispatch market."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For example, that ramping up a generator is twice as expensive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n.generators.loc[n.generators.index.str.contains(\"ramp up\"), \"marginal_cost\"] *= 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or that generators need to be compensated for curtailing them or ramping them down at 50% of their marginal cost."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n.generators.loc[n.generators.index.str.contains(\"ramp down\"), \"marginal_cost\"] *= -0.5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this way, the outcome should be more expensive than the ideal nodal market:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "n.optimize(solver_name=solver)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Costs are now 502 k€ compared to 301 k€."
   ]
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "kernelspec": {
   "display_name": "",
   "language": "python",
   "name": ""
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
